{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# **ReWOO (Reasoning WithOut Observation)**\n",
        "\n",
        "ReWOO (Reasoning WithOut Observation) is a novel prompting paradigm designed to enhance the efficiency and accuracy of Augmented Language Models (ALMs) by decoupling reasoning from observation-dependent interactions with external tools. This approach mitigates issues such as high computational cost and token redundancy prevalent in conventional ALM methodologies.\n",
        "\n",
        "### Key Concepts of ReWOO:\n",
        "1. **Decoupled Reasoning and Observation**: Unlike traditional ALMs, which require LLMs to halt their reasoning process to fetch external tool responses, ReWOO enables models to generate a structured reasoning plan before interacting with external data sources.\n",
        "2. **Modular Plan-Work-Solve Framework**:\n",
        "   - **Planner**: Constructs a comprehensive plan outlining logical steps required to solve the given task.\n",
        "   - **Worker**: Retrieves the necessary information from external tools based on the Planner’s blueprint.\n",
        "   - **Solver**: Synthesizes retrieved evidence and plans to generate the final answer.\n",
        "3. **Token Efficiency**: ReWOO reduces token consumption by eliminating redundant context repetitions, achieving up to 5× token efficiency compared to interleaved reasoning paradigms like ReAct.\n",
        "4. **Scalability and Specialization**: By decoupling parametric reasoning from non-parametric tool calls, ReWOO allows for instruction fine-tuning, enabling smaller models like LLaMA 7B to approximate the reasoning capabilities of larger models such as GPT-3.5.\n",
        "5. **Robustness Against Tool Failures**: In scenarios where external tools fail or return incomplete results, ReWOO remains resilient by structuring logical inferences before calling tools, reducing dependency on external observations.\n",
        "\n",
        "### Advantages of ReWOO:\n",
        "- **Lower Computational Overhead**: Reduces prompt redundancy, cutting down costs in commercial LLM APIs.\n",
        "- **Improved Accuracy**: Achieves a 4% improvement in reasoning accuracy on complex tasks like HotpotQA.\n",
        "- **Enhanced Generalization**: Facilitates the transition of reasoning abilities from large models to smaller, fine-tuned models, making ALMs more accessible and scalable.\n",
        "- **Better Multi-Step Task Handling**: Suitable for multi-hop reasoning and fact-checking applications that require structured workflows rather than continuous interruptions.\n",
        "\n",
        "ReWOO marks a significant shift in how LLMs interact with external knowledge, optimizing efficiency and enhancing performance for real-world AI applications.\n",
        "\n",
        "Paper link: https://arxiv.org/pdf/2305.18323\n",
        "\n",
        "Code Reference: https://github.com/langchain-ai/langgraph/blob/main/docs/docs/tutorials/rewoo/rewoo.ipynb\n",
        "\n",
        "![2.png]()"
      ],
      "metadata": {
        "id": "WsD629yiOqe7"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "87P8jwY5sohm"
      },
      "outputs": [],
      "source": [
        "# install dependencies\n",
        "!pip install --q langgraph langchain-community langchain_openai"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "from google.colab import userdata\n",
        "os.environ['OPENAI_API_KEY'] = userdata.get('OPENAI_API_KEY')\n",
        "os.environ[\"TAVILY_API_KEY\"] = userdata.get('TAVILY_API_KEY')"
      ],
      "metadata": {
        "id": "qn69IZCbtPiI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from typing import List\n",
        "from typing_extensions import TypedDict\n",
        "\n",
        "class ReWOO(TypedDict):\n",
        "    task: str\n",
        "    plan_string: str\n",
        "    steps: List\n",
        "    results: dict\n",
        "    result: str"
      ],
      "metadata": {
        "id": "fjF_MqE5tRWm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_openai import ChatOpenAI\n",
        "llm = ChatOpenAI(model=\"gpt-4o\")"
      ],
      "metadata": {
        "id": "kZURxItata3g"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain_community.tools.tavily_search import TavilySearchResults\n",
        "search = TavilySearchResults()"
      ],
      "metadata": {
        "id": "ULsWJDDUtyIR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "prompt = \"\"\"For the following task, make plans that can solve the problem step by step. For each plan, indicate \\\n",
        "which external tool together with tool input to retrieve evidence. You can store the evidence into a \\\n",
        "variable #E that can be called by later tools. (Plan, #E1, Plan, #E2, Plan, ...)\n",
        "\n",
        "Tools can be one of the following:\n",
        "(1) websearch[input]: Worker that searches results from websearch. Useful when you need to find short\n",
        "and succinct answers about a specific topic. The input should be a search query.\n",
        "(2) LLM[input]: A pretrained LLM like yourself. Useful when you need to act with general\n",
        "world knowledge and common sense. Prioritize it when you are confident in solving the problem\n",
        "yourself. Input can be any instruction.\n",
        "\n",
        "For example,\n",
        "Task: Thomas, Toby, and Rebecca worked a total of 157 hours in one week. Thomas worked x\n",
        "hours. Toby worked 10 hours less than twice what Thomas worked, and Rebecca worked 8 hours\n",
        "less than Toby. How many hours did Rebecca work?\n",
        "Plan: Given Thomas worked x hours, translate the problem into algebraic expressions and solve\n",
        "with Wolfram Alpha.\n",
        "#E1 = WolframAlpha[Solve x + (2x − 10) + ((2x − 10) − 8) = 157]\n",
        "Plan: Find out the number of hours Thomas worked.\n",
        "#E2 = LLM[What is x, given #E1]\n",
        "Plan: Calculate the number of hours Rebecca worked.\n",
        "#E3 = Calculator[(2 ∗ #E2 − 10) − 8]\n",
        "\n",
        "Begin!\n",
        "Describe your plans with rich details. Each Plan should be followed by only one #E.\n",
        "\n",
        "Task: {task}\"\"\""
      ],
      "metadata": {
        "id": "j88-9OuqtdQf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "task = \"what is the exact hometown of the 2024 mens australian open winner\""
      ],
      "metadata": {
        "id": "AIBjD7BGtgYU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "response = llm.invoke(prompt.format(task=task))"
      ],
      "metadata": {
        "id": "DLi7E1obtkQD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(response.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wvXEwjlmtn-_",
        "outputId": "df66f968-4117-4d16-ec5a-97f0fc083243"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Plan: Begin by identifying the winner of the 2024 Men's Australian Open using websearch. This will provide the name of the player who won the tournament.\n",
            "#E1 = websearch[\"2024 Men's Australian Open winner\"]\n",
            "\n",
            "Plan: Once we have the winner's name, search for the winner's exact hometown using websearch, as this specific information might not be widely known but potentially available from reliable sources like the player's official profiles or other reputable databases.\n",
            "#E2 = websearch[\"hometown of 2024 Men's Australian Open winner #E1\"]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "from langchain_core.prompts import ChatPromptTemplate\n",
        "\n",
        "# Regex to match expressions of the form E#... = ...[...]\n",
        "regex_pattern = r\"Plan\\:\\s*(.*?)\\s*(#E\\d+)\\s*=\\s*(\\w+)\\[(.*?)\\]\"\n",
        "prompt_template = ChatPromptTemplate.from_messages([(\"user\", prompt)])\n",
        "planner = prompt_template | llm\n",
        "\n",
        "\n",
        "def get_plan(state: ReWOO):\n",
        "    task = state[\"task\"]\n",
        "    result = planner.invoke({\"task\": task})\n",
        "    matches = re.findall(regex_pattern, result.content)\n",
        "    return {\"steps\": matches, \"plan_string\": result.content}"
      ],
      "metadata": {
        "id": "Uu5y0CoQtqfU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def _get_current_task(state: ReWOO):\n",
        "    if \"results\" not in state or state[\"results\"] is None:\n",
        "        return 1\n",
        "    if len(state[\"results\"]) == len(state[\"steps\"]):\n",
        "        return None\n",
        "    else:\n",
        "        return len(state[\"results\"]) + 1\n",
        "\n",
        "\n",
        "def tool_execution(state: ReWOO):\n",
        "    \"\"\"Worker node that executes the tools of a given plan.\"\"\"\n",
        "    _step = _get_current_task(state)\n",
        "    _, step_name, tool, tool_input = state[\"steps\"][_step - 1]\n",
        "    _results = (state[\"results\"] or {}) if \"results\" in state else {}\n",
        "    for k, v in _results.items():\n",
        "        tool_input = tool_input.replace(k, v)\n",
        "    if tool == \"websearch\":\n",
        "        result = search.invoke(tool_input)\n",
        "    elif tool == \"LLM\":\n",
        "        result = llm.invoke(tool_input)\n",
        "    else:\n",
        "        raise ValueError\n",
        "    _results[step_name] = str(result)\n",
        "    return {\"results\": _results}"
      ],
      "metadata": {
        "id": "tjnpki_Kt3Pa"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "solve_prompt = \"\"\"Solve the following task or problem. To solve the problem, we have made step-by-step Plan and \\\n",
        "retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might \\\n",
        "contain irrelevant information.\n",
        "\n",
        "{plan}\n",
        "\n",
        "Now solve the question or task according to provided Evidence above. Respond with the answer\n",
        "directly with no extra words.\n",
        "\n",
        "Task: {task}\n",
        "Response:\"\"\"\n",
        "\n",
        "\n",
        "def solve(state: ReWOO):\n",
        "    plan = \"\"\n",
        "    for _plan, step_name, tool, tool_input in state[\"steps\"]:\n",
        "        _results = (state[\"results\"] or {}) if \"results\" in state else {}\n",
        "        for k, v in _results.items():\n",
        "            tool_input = tool_input.replace(k, v)\n",
        "            step_name = step_name.replace(k, v)\n",
        "        plan += f\"Plan: {_plan}\\n{step_name} = {tool}[{tool_input}]\"\n",
        "    prompt = solve_prompt.format(plan=plan, task=state[\"task\"])\n",
        "    result = llm.invoke(prompt)\n",
        "    return {\"result\": result.content}"
      ],
      "metadata": {
        "id": "2noRsv7Yt9Pr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def _route(state):\n",
        "    _step = _get_current_task(state)\n",
        "    if _step is None:\n",
        "        # We have executed all tasks\n",
        "        return \"solve\"\n",
        "    else:\n",
        "        # We are still executing tasks, loop back to the \"tool\" node\n",
        "        return \"tool\""
      ],
      "metadata": {
        "id": "lE2IfcieuA-Z"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from langgraph.graph import END, StateGraph, START\n",
        "\n",
        "graph = StateGraph(ReWOO)\n",
        "graph.add_node(\"plan\", get_plan)\n",
        "graph.add_node(\"tool\", tool_execution)\n",
        "graph.add_node(\"solve\", solve)\n",
        "graph.add_edge(\"plan\", \"tool\")\n",
        "graph.add_edge(\"solve\", END)\n",
        "graph.add_conditional_edges(\"tool\", _route)\n",
        "graph.add_edge(START, \"plan\")\n",
        "\n",
        "app = graph.compile()"
      ],
      "metadata": {
        "id": "I4Re1UlHuD7Y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import Image, display\n",
        "display(Image(app.get_graph().draw_mermaid_png()))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        },
        "id": "L5p0yCgmuFso",
        "outputId": "5dba12b9-773f-4fa0-834d-8cc053166af1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIEAAAGwCAIAAAABk5G7AAAAAXNSR0IArs4c6QAAIABJREFUeJztnXlcVNX//8+dDYbZ2TdBEDREEBUVcEk0NAjX/KmpYZaaVpaVmfUxP2RlqWlmfj99LE3NfWtRc99FRU0EZQdF9mUGmJ1Z7/39MX6Qjw4wy505d+7nPh/9YXfO8h5ec8659573eb8RDMMABVRosA2goDQgAJQG8KE0gA+lAXwoDeDDgG0AaGnQK6V6lcygUaM6DQrbHItgudPoDMSDT+fwGf6h7oh9v2QE1vNB7QNNxX3lw3ylXwhbqzZyBAy+F/wfhIW4udOlEp1KbtS1oTVl6pDnPMJjuH2H8BG6La1B0KCxUnPtWLPQm+npzwqP4fC9mE42AHcqC9UP85XVJeq+CYL4FJG11Z2tweUj4qYqTdJ476AItjP7dQ43/mq+d1X24hz/0CgPy2s5TwONCt23tnLMDL8Qa+xzOXQa9MKBJp8gt0EvWDognKSBXovuXPXolWWhHIFNU6arceOvZjaHHjdKaElhZ2iglBoObKh5Y1VPR3dEKK4dbdZpjMnTfLst6Yzng31rq2YvD3FCR4Ri2AQvGg25nyXrtqTDNTi/r2nCm4FuHv+LD4PPT/VpqtbWV2i6LubYP83D+yqN2ugX6u7QXohMzHDB1d/FXZdxrAbXj0mSxns7tAuC49vDjefJLM9TdlHGgRqU5igj4rgiX2c8ghmNxtzcXFjVu2b4RO/SO4ouCjhQg7IchV+Ik2ahL774YvXq1bCqdw1PxJBJ9JI6XWcFHKhBRYEqrB/Hce13RKvV2lbRdGtuc3ULCY/hVuR3Oh056jVZZZE6OpHviJazsrJ++OGHmpqawMDAqVOnTp8+PTMz8+zZswCA+Ph4AMDRo0cDAwNzc3O3bt1qmmGio6OXLFkSFRUFADh37tzy5cu//fbbXbt2FRQUzJkzp7Gx8dnq+NocEce9ebK5s08dpUFro47phv8gU6vVH3/8cXh4+IoVK8rLy8ViMQDg9ddfb2xsrK2tXbVqFQDA29sbAFBXV6fVaufNm0ej0Q4dOvTuu+8eO3bM3f3x3LhmzZq333570aJFISEhGo3m2er4wvdkVJeqO/vUURooZQYOH//GW1patFrt6NGjU1NT2y+GhIQIhcLm5ua4uLj2i6mpqWlpaaZ/9+3bd+HChbm5uQkJCaYr06dPT09Pby/8bHV8YbrRMBQYdBiDhTz7qaM0UMuNvsFuuDcbFBQUGxu7bds2Nps9ZcoUFovVWUkEQS5evLh79+6KigoPDw8AQHPzk9lgyJAhuNvWNR58ulphMPui3lFrMo2O0Bn4N44gyKZNm9LT0zdu3DhlypScnJzOSm7duvWjjz7q27fvhg0blixZAgBA0SebdCZVnIk7m452sknoKA3c2DSlVO+Ilrlc7vLly48cOcLlcj/44AO1+vE82/Hlo1ar3b59+6RJkz788MO4uLiYmJhum3X0u8tWsY7DN//O2FEaePDpKrnRES2b7iODgoJmzJihVCrr6uoAAGw2u7m5uf2X3tbWptVqTTdCAACpVPrUOHiKp6rjjkGPoSjo7CbFUeuB0JvVWNnNuyob0Ov1L7/8ckpKSq9evQ4dOsTlcoODgwEAAwcOPHr06OrVq+Pi4vh8/siRIyMiIvbv3+/l5aVUKn/66ScajVZeXt5Zs89Wx9fsNoUhNKrTRyV6ZmYmvv2Z4ImYp3+tj0/xxLdZlUpVVVV18eLFCxcu+Pj4ZGZmmjSIiIiQyWSnTp3KyckRCoVDhgwZOHDgtWvXDh48WFlZuXjx4tDQ0CNHjsyaNauysvLcuXPTpk0TCp9ssDxbHV+zC7MVdDoI6WN+EXLgHs7vm2uHvOhJyn1jazmyqSYp3Tsg3PybGwe6k0QO5DU80nShQV5e3nvvvffsdR6Pp1CYf8n13nvvTZ48GVczzTBv3jyzE5efn19jY+Oz12fNmjV//vzOWtNpMAaT1pkADt/L/OmTh6+t7Mlim1+LtFptx3t2SxAIBByOw99BicVivd7MTZ1er2cyzdzg83g8Ho/XWWuXj4hFvqzYEYLOCjhWg/zrMnGN1pI9VbKilBoObayZm9nVXrpj93D6JQnUCqNC6pCbVJfg3hXZiMk+XZdx+DbvmFf89q+rdHQvxCT3khRFsYj+3UyeDtfA3YOWNjfg8MYaR3dENMpylI8KVcMndf8W1kk+XtIm/bl9jVPfC3ZCX0Sg5G9FVbE6ZbafJYWd5HIi9GUmvuT186cPFS0G5/QIkZsnWyqLLBXA2T6/WjV6fn+juwc9aby3O4eEHkelOYrrx5rjnhda6OVoAoLve2G2/PoxSf+RQr+e7p09vrsW8hZDRb7yUYHKnUsfNt6bK7TuyRfaGZDCm/Kyu8racnXscCGGAY6AzhUyaS7iEMxk0uQtepXcoFWjtQ/a9Fo0rB+nb4LAO7DTPaUugKaBCdSIVRW3ySQ6ldyo1aAaFc5PEnK5vKampm/fvvg2yxUwUCPmwadzhQzfHu5eAbb86duBrIGjuXPnzpYtW3766SfYhnQFCRdGl4PSAD4k14BOpwcEBMC2ohtIroHRaKyvr4dtRTeQXAMajeZ8NxZrIbkGKIq2O78QFpJrQKPRRCKrD207GZJrgKJoa2srbCu6geQa0Ol0k/MLkSG5BkajsaaG6NtHJNfAJSC5BjQazQm+MHZCcg1QFFWpVLCt6AaSa4AgCJ/vkGNxOEJyDTAMk8vlsK3oBpJr4BKQXAMajebv7w/bim4guQYoijY0NMC2ohtIroFLQHIN6HQ67ofucYfkGhiNRtOhQSJDcg1cApJrwGAwqPemkDEYDNR7U4ruIbkGlG8LfCjfFgqLILkGlH8RfCj/IvhQ703hQ703pbAIkmuAIIhA0GmwDoJAcg0wDJPJus9AABeSa8BgMIKCgmBb0Q0k18BgMNTW1sK2ohtIrgH17ho+1Ltr+NBoNC8vL9hWdAM5z4hPnz5do9FgGKbValUqlZeXF4ZhbW1tphj7RIOc42DMmDF1dXV1dXXNzc0ajaa2traurq6LsH9wIacGM2bMeHYpHjduHCRzuoGcGvD5/BdffLHjlaCgoFmzZsGzqCvIqcGzQ2H8+PFcLheqRZ1CWg0EAkF7HpDg4OBp06bBtqhTSKsBAGDatGk9evQAAKSlpRH5JAjh0qdr21BxrRanYFKMscMybt68mRg7uetMfRZCoyMiH5bID+ese8R6Pji7u6miQBnYy4NANnWAK2DUlKo4Akbc88LwGNyOGhJFA9SIHfmhNmqoKLQv0Y9RYkZwdk/dwGRhWD98vAWIsh78trl2QLI38QUAACB0MDYj8O9zLTVlbbg0SAgNHuQpPf3c/Xq6UorfpAl+dy9KcWmKEBqIa7Uul+Sa78WsLFbhMpET4ptr1Cjfy67wlFAI6MmWS3DIP0YIDfQaFDUS4tbAKlRyAzCTetFqCKHB/ziUBvChNIAPpQF8KA3gQ2kAH0oD+FAawIfSAD6UBvChNIAP2TSQyaTJY+L/PHoYtiFWQDYNXBFKA/gQzq/CEsrKSxa8OWvs2JcKC+83NtYHB4fMfGXuC2NefLZkU1Pjtu3/unnzmkql7NEjtGOx8RNHLXnvk6ysi9k3szgc7vj0l+dkdJoL3KG48DhoaKj74P1Pv/ryu6DAHl+tXnHp8rlnyxiMhuLigokTpi56cwmfL/hq9Yqi4oL2T79Z88+IiD4bv/s55YW0HTu3ZGdnOfcbPMYlx4GJGdMyBsTFAwAGDRwy941p+/btGPX8C0+VCQwI2vHLIQRBAACpqRMnv/zCtWuXop6LNn2aljpx1sy5AICIXr3/OvHHrb9vJCQMd/4XcWEN2qHRaPHxCb//fkCvN7OzWP6gdMfOLSUlhaYwLi0tze0fubuzTf+g0+k+Pr7NErETrX6CC89FHeFxeRiGtWmedjbJuXv7rbfn6HW6ZR/98/N/ruXzBSiGmm2BQWcYUTg558kwDgAAYnGTu7s7n8eXyf7L32TXrq2BgcGrv9rIYDAAAOz//PAJBRnGgUKpuHr1Qr/o/gAABoMJAFAoHscZl8mlEb16mwTQ6XTqNjWKmh8HEHHhcbB77y+SZnFbm/ro0cMqtWruawsBABwOJygw+OCh3QKBcHz6lLi4+NOnj504+SefJzh0ZI9CIX9U8QDDMNMqTRBceBxwuby9e7dv3fZ/XC7vqy+/69s3xnT9H//4Kjg45PSZ4wCA119bNDg+8YfN6zZtXjto4NDMlWuaWyR3c/+Gbft/QQif37O7G31DPML7W3pmz/SMtvrL7xITRzjYtK74/YfKiQsDBd72usK78DggDZQG8HHJNTkyos/F88Sa0+2BGgfwoTSAD6UBfCgN4ENpAB9KA/hQGsCH0gA+lAbwoTSADyE08OAzEDpsI6xH4M2iM3D4AxJCA56IIa7WwLbCOrRqo7i6jSvE4bdDCA169PFQSnE4bO1MGio0fQbjExOJEBqIfJkR/bmXDxE9UUE7zXXauxckIyZ549IaIfbRTJTmKHMvScP783yC2UwWgfZ726HRkOYGrUqqL70je2VZCJ2Bj5EE0gAAIKnV3r8mU7QYpGIbpyaj0aDVaj08zMfg0et1arVaIBDa1rhngBsCsOBIj/7P45pTASMXu3fvXr9+fWeffv/994MHD163bp1zjeoGQqwHOFJUVBQVFdXZpzk5OUaj8fjx44QKukw2DYqLizvToK6urrW1FUEQpVK5adMm4sSDJ5UGWq3Wz8+vZ8+eZj8tLCwUix979dbW1i5btsy51nUKqTTIz88363ptIisrS6vVmv5No9HKy8szMzOdaF2nkEqDioqKoUOHdvZpQUFBRxdHFEWvXLmyd+9eZ1nXKaTS4N69e11kBVQqH4eZNd2NsFgsFos1c+ZMJxpoHpf0L+oMnU7XxU2RXC739/c/fvx4Tk6On58fcfJFEesZzR5QFB06dOjt27e7Lblr167m5uYlS5Y4xa7uIc9cVF5enpycbEnJpKQkQuUEIc9c9ODBAxbLogCdvXr16tWrl+MtshTyjINHjx519mTwLH/88Uf7fSp0yKOBQqGIjIy0sPDx48eLioocbJGlkEeD3Nxcy9NVT5o0ycHmWAF51oOqqqqQkBALC6enpzvYHCsgyThobW0VCoVstqVHXx89enTp0iUHG2UpJNGgsbFRKLRiZ0Yul+/cudORFlkBeTQIDw+3vHxkZCRxMkWRRAOxWGz5RAQAYLPZqampjrTICkiigVQqFYlEVlX55ptvDAaDwyyyApJooNPp/Pz8rKpy6dIlqRSfZCp2QhINxGIxnW6dy9s777xj4bsNR0OS5wONRuPubl1KI+I8IpBkHPB4PGtTkh47dkwikTjMIisgiQZisdjaBfbgwYNNTU0Os8gKSKIBgli9GZWYmOjp6ekwi6yAJOtBQECAKVCU5bz11lsOM8c6SDIO5HK5TCazqkpHVxe4kEQDDoejVqutqvLxxx8TZC+dJBr4+/tbFajOYDAkJSVZezvrIEiiAYvFqq6utrw8g8FYt26dIy2yApJo4O3t3e7CZQkKhSI/P9+RFlkBSTTw8vKqra21vPyNGzf27NnjSIusgCQa+Pv7NzRYcZyNwWAkJiY60iIrIM/zgVXvTUePHu1Ic6yDJOOARqOJxWLLj3VkZ2cT5GUReTQAAAwdOtTyJeGLL74gyAYOqTTw8PAoKSmxpKRer09ISLDcGcnRkEeDqKgoC/fFmEzmZ5995niLLIU8GoSFhV2/ft2SkrW1tXl5eY63yFLIo0FkZKRAILDkjcWhQ4fu3bvnFKMsgiT3piZQFE1LS9Pr9a2trREREQcPHjRbLCwsbODAgU63rlPIoMHIkSOVSiWNRmtPbECj0RISEjorP3HiROca2A1kmIsSExOZTKZpN810hcPhdPZL12g0Fy9edK6B3UAGDdasWRMWFtbxipeXV2xsrNnC2dnZf/31l7NMswgyaAAAWLVqVbvjO4ZhgYGBne0V+/j4vPHGG861rhtIokHv3r3nzZvX7nodHx/fWcno6Oguzs9CgSQaAADS0tJSU1Pd3d19fHz69+/fWbFvv/1WoyFW7DyC3hehRkzRagTAuv3eeXPerX7Y3NjY2DOor0xiJnBFU1PT7esF2tfoWqXVMarodBpX5JDok4Q7I16Rr8q9LK2vaPMOdG9TWp+4D8NAF3m3MAzDMIRmy+gX+rKaqtp6D+I//zI+YezaIZYGxbeVRbfkCem+XCERB6i2DW2sbPv7tHj2J6F0Jm4R9wikQdEtRdldZfKMANiGdENrk+7S/vqMz0LxapAoazJqBEU35cQXAAAg8mVFJQjvXsTt7AJRNJDUanVawmWy7AyOgFFT/nSOWpshigZSiS4gzAO2FZYi8nVDAG7rAVE0MBowW+6CIIFiWGsjbr6qRNHgfxlKA/hQGsCH0gA+lAbwoTSAD6UBfCgN4ENpAB9KA/hQGsDHtTVoaKivb6izpwWZTJo8Jv7Po4fxM8pqXFiD2rqambMnlJQUwjbEXlxYA6PBQJxNQHsg4ratJTQ1Nc6ZOxUA8Pmq5Z8DMG5c+vJlmQCAwqL8f2/ZWFJS6O7OTkocuWjR+3we33QofPuOf58+c1wmk4aGhr02583hw0bB/hKPcdVxIBJ5/uPTLwEAc19buGnj1tkzXwcAPHr08MOlC/V6/bKP/jnn1flZWRc///xjU/lv13954OCu9Jcm/+PTL/39Az9bufTevbuwv8RjXHUcMJnM3pHPAQBCQnrGxMSZLu7es41Go61ds5nH5QEAeDz+6m9W5uXliESep88cz3h13mtz3gQAPD9yzOyMyTt2btmw/t+wvwdwYQ3Mkpt3Z8CAwSYBAACDBycCAEpKC9lsDwDA8OGPsyMgCDI4PuHsuRNQjX2Cq85FZlGplELBkwibPB4fACCRiFUqJQBAJHziBcznC9RqtUqlgmTpf0EqDby9feXyJ1GMWltbAABcLs/b2xcA0PGjlpZmBoNBxW2xFzc3dwBAs0TcfiU6OjY37067S++VK+cBADExcVFR/RAEyb6ZZbqu0+myb2ZFR8fS6XQGgwkAUCjkkL4EcO31wNfXLzAg6ODh3e5stlwumzJ5xuyZr1+4cPrjTxaPT3+5qalh568/DYiLj+s/CEGQcWPTd+zcYjQaAwOD//rr95aW5k8/+cJ0YicoMPjgod0CgXB8+hQoX4ROkFx5klqtTGIIec58zl2zIAjSt2/srdvXL1w8Xd9QN3xYcmBgcEy/Abf/vnHs+JGS0qLkUWM/WrrSzc0NADA4PlGlUp489eeFC6c5HpylH64wrdgAgKi+McXFBQ8flqWlWnpOTduGVtxX9B9pYw7gp78IQR41i27JK4s0wyb5wjbEIuQt+gt76l5dgY/LqQuvB6SB0gA+lAbwoTSAD6UBfCgN4ENpAB9KA/hQGsCH0gA+lAbwoTSAD6UBfIiiAYNFY/McEpHDESA0xDMAt9xqRNFA5MOqLSPE7q4ltNRruwhMYi1E0cA7iOXOoWMuclJf2aoPjsTtRDtRNAAADEgWnd5hadBwiFQVqaqKFLEjBHg1SJR9NBN1DzSXD4uHvOTL92S4cwi3PLQ26sTVmop8+cvvBuM4FxFLAwCAuEabc15aXaZmutPUsv8OzY4BFENpNkWAshzTHwR55m/sE+ym16KRcbz4sdYlCe4WwmnQjkGHPfV3ePXVVzMzM3v16uXorpctWzZ+/PgRI0Z0vEhjPKsLPhBXg6eQSCQIgnh5eTmnu9LS0t69ezunLwKtyV2g1Wo1Go3TBAAABAYGWpVgxx5cQ4OlS5damw7TTrhc7oEDB86cOeOEvlxgLsrPz+dyuT179nR+1+fOnRs1apSj5Se6BhiGYRjm6HuhzkBRFEVRR2tA9LnohRdekMuhOeTSaLSvv/76jz/+cGw3GIHZuXPnhQsXYFuBzZ07V6/XO659os9F/wsQdy7auHGjVWlIHcquXbsePXrkoMYJqsHmzZuFQiGXy4VtyGOGDx++dOlSBzVOxLnIaDQqFIr2ZAYEQSaTMRgMDseKExIWQkQNJBIJn89nsXDbqCI4hJuLTp069d133xFTgM2bN+/duxf3ZgmnQVFREaESKXZkwYIFOTk5uDdLxLnofw0CjQOdTvfFF1/AtqIbVCrVzp078W2TQBp8//33kZGRsK3oBg6HU1BQcP78eRzbJMpcZDAYGhoagoODYRvSPc3NzSUlJUlJSXg1SCANEASh0wm3j+8ECDEXqVSq0aNHu5AAly5dOnXqFF6tEUKD06dPf/DBB7CtsIKoqKhNmzbh1RpR5iKXo7W1lcvlmvLV2gn8cVBXV+eIBx9HIxQK8Zo84Wuwfv16iDtlNlNXVzdp0iRcmoKsgV6vDw8PHzWKKCEWLScoKMjLy6u6utr+pqj1AD6Qx8GZM2dKS0vh2mAzRqNRq8UhQxdkDTZs2CAS4exC6zTa2tqWLFlifzswNVCr1StWrPDx8YFogz1wuVwMw2pq7D0zQa0H8IE5Di5cuHD9+nWIBtiPRCKpr6+3sxGYGhw/flyvtzqnOqF4+PDhqlWr7GwEZmzNKVOmDBw4EKIB9hMVFcXn8+1shFoP4ANtLlKr1Z9//jms3nGkuLjYzrjZ0DQQi8V5eXmweseRX3/9NSsry54WoGnA5/M//PBDWL3jyMCBA+18WqbWA/hAGwcFBQW//vorrN5xRKlU2umSDU2D+vr6goICWL3jSH19/fLly+1pAdrzQVRUlKenpwUFiU5AQEBoqF3Bx529HixatCg7O5tGoyEI8vgkEIIEBQUdPXrUmWYQCmfPRRkZGZ6enqaoAwiC0Gg0DMOGDRvmZDPwJTs722AwWFDQPM7WIDExsXfv3h0HX48ePV555RUnm4Eva9eura2ttbk6hDU5IyOjfc8Aw7CkpKSQkBDnm4EjI0eOtGdKh/N8sHjx4uvXryMIEhwcvHnzZpdwM3UccO5NTUMBw7DExEQSCFBRUdHS0mJzdTgaDB48uHfv3v7+/rNmzYJiAL7s3bv30qVLNlfvZi5qqtbmXJQ2VWpUctvXfbOYbkxxD0ThG+JOQ0DEAG7MMNzizXXLvn37hEJhamqqbdW70qCiQJ19orn/814iPxab6xpO0SgKJLWapqo2lVQ/LsMPtjkW0akGhdnykjuqF2YHON0kfCi4Lm2qUk94M9AJfdXX1xsMhh49ethW3fxU0KZCS+8qXVcAAEB0klDk515yW+GEvq5cubJv3z6bq5vXoKGizVEB9JwIz5NZWaJ2Qke9evUKCwuzubr5d3byFoN/T7YdVhECz0C3hofOiN8cHx8fHx9vc3Xz40CrNuo0LhL3uHMQDLQ06JzQUUNDw927tmeGh3/+gASUlJTs2rXL5uqUBjgQHBw8ePBgm6u7cP5k4tCrVy97og9T4wAHJBKJPe4tlAY4UFNTs2PHDpurUxrggLe3tz1bgZQGOBAcHDx37lybq1Ma4IBUKr169arN1SkNcKChoWHLli02V6c0wAGhUGhPKB1KAxzw9/d/6623bK5OaYADSqUyOzvb5upwNJj7xrRVX3wCpWtH0NjYuGHDBpurU+MABzgcTlxcnM3VKQ1wwN/f/9NPP7W5Om4a7N23Y9qMtNSXhi9+7407ObdMFwuL8t9dMm9catLEyWPWrP1crng6Ro5Wq50wafRXq1e0X8nNvZM8Jj47OwsAUN9Q99nKpWnpIyZNeWHZx+8UlxTiZS2+aDQae/z48dHgTs6tn7dujo0d+MGST/39AtrUagDAo0cPP1y6UK/XL/von3NenZ+VdfHzzz9+qqKbm9vYlJeyrl1Sqx9vOp49d8LPz3/IkKTmZsnid1+XK2TvvL30zQXv6vX695bMq6h4gIvB+FJbW2vP+UZ83l03NNQBACZPnBYdHZuSkma6uHvPNhqNtnbNZh6XBwDg8firv1mZl5fTv/9/nUkenz7lyG/7rl69MG5cularvXL1/PRpGTQabdfurSKh5/p1P5rS0aS8kDY7Y9LxE78vfttRMfBtxs3NLSIiwubq+GiQMHQ4j8df/fVni9/5KCFhuOlibt6dAQMGmwQAAAwenAgAKCktfEqD0NCwmJi4c+dPjhuXfu36ZY1Gk5Y6EQBw8+a1JnFjWvqTZH16vV4ibsLFYHwJDg5evXq1zdXx0cDLy3vzpl/+78cNn/xjSb9+/Veu+NrHx1elUgoFT+Li8Hh8AIBEIn62+viXpnyzNrO5WXL23Inhw0Z5enoBAFpamxMTRyyYt7hjSQ6HKFkpOqLX65uamoKCgmyrjtuaHBLSc83Xm9Z/+2NFRfmatZkAAG9vX7lc1l6gtbUFAMD9z7DoyMiRYzgc7m+/7799+8aECVNNF3k8vkwmDQnp2fE/Ly9vvAzGkaqqqvfff9/m6rhpoNPpAAADBwxOSBhRWlYMAIiOjs3Nu6PRaEwFrlw5DwCIiYkDALCYLEWHeyQ3N7eUlLR9+3cGBfUYEPfYSWTgwCH5+XklpUXtxdra2vCyFl8YDIY9eSTpmZmZz16tLW8zGoB/mKUuRkXFBUven28wGB48LDt+/Lfn+vRNSUnrGRp+5Ld9uXl3mExW9s2sbdv/FRszYE7GfARBiosLLl85r1IpB8TFmyJU+vn6//HnodmzXu/bN8bUZnh45NlzJ86ePWE0GqtrKvfs+eXy1fOjk8dZ/t00KmNVkTJmuMOdf4VCYXp6us3V8VkPWExWaEjY3r3bMQzrHzfo3XeWAQCCg0PWfrP5p60/rF33OZvtkfJC2sI3l5jc9+a98bZCIT916uicjAWmMK09e4bHDxo6duyTbxIUGLx50y8/btm4Z+8vCIJERj43edJ0XKzFHaPRqFareTwz06wlmPf5vXWqRasBccmufXZV2qS7eqRh5nKHH7QqLy9fsWLF/v37batOvavABzc3N5vrUhrgQEREhD3JQSgN4ENpgAPFxcX2HKyjNMABo9GyH3NPAAAME0lEQVRoTwx4SgMciI6O3r59u83VKQ3wgRoHkMnNzV24cKHN1SkNcMCeoC3U+QN8GDRokD3BcikNcABBEHuOsVJzEQ6cO3fu44+f3iq3HPPjgMmikSDiJkJD+F445M7qFhRFbX5p2qkGHAG9/p4zTlc7FJlYhzhlnI8dO3bs2LE2Vzdvo1egG4a6/EhQKQyB4c446a7RaOwJed2JBgEsvicj96LtcZGgo5QaCq+3DkgWOqGvAwcObNu2zebqnY7VEZO9MRS7fUricgf2MQzUlqtPba+Z9YldYUctx8PDIzDQ9gAx3cSQuntRev+aDEMBm4Nz/CIUwzAMo+MdQ4ojZFQUKKOHCpKnu0yqo+5jCmIYUEoNKhnOcbxu3bqVk5NjzyO+WRhMmncQC982u0Uikbi7u3O5Nvo+df+MhiCAJ2LwRDg/zbmXanS0Jv+e7vg2C4X169cnJyfbfGtEPaPhAJ1OtyfTHrR3FXQ6nc12+RBJJr788kt7qkMbByiK2plGhjjI5XIUtf3uEZoGbm5urpul8SkmTJiA/zOaE0BR1P4Me0TAaDTyeDx73hdB04DFYrluxtiO0On0Y8eO2dMCNA2YTCYuSbiho9Vq7RzQ0DRgs9mE9WW3imvXrtlzOBmmBjwez557CeJgNBoHDBhgTwvQng9EIpH92Z+JQEpKip0twMwTGBYWRoKhIJFI2s8a2QbMdxUtLS11dXUQDcCFBQsWNDY22tMCTA0CAwPtSeVDEAwGg5350WBqEBsb29zcDNEAXLA/sxtMDTw9Pe/fvw/RAPtRqVRisZkT11YBU4PIyMiysjKIBtjP1q1bT548aWcjMDXo06ePq2dZwDAsISHBzkZgauDh4aHRaAoLCRoRxxKWLFnSu3dvOxuBvI82aNCgO3fuwLXBZqRS6Y0bN+xvB7IGiYmJDx8+hGuDzezfvz8/P9/+diBrMHTo0HPnzrUHkHItMAx78cUX7W8HTr7MjmRmZg4aNGj8+PFwzYAIfL+K9PT0mzdvwrbCaioqKuzJgdMR+BrEx8cXFxdXVFTANsQ6vvzyS7xurOFrAACYMWOGzQE3oKBUKidOnGhPTNOOwF8PTEyYMOHAgQOk8TiyCkKMAwDAzJkzN2/eDNsKS5k3bx6O93JE0WDGjBk3b95sbW2FbUj3/PHHH6GhoR4eHng1SJS5CABw8uTJa9eu2ek36ATEYrFIJDJFXcUFoowDAEBqampdXV1eXh5sQ7rCaDTy+XwcBSCWBgCA5cuXHzhwALYVXTF16lQ7dy6fhVga9O7d28/P79dff4VtiHn+/vvv2bNnh4TgHCCPQOtBO2lpadu3b/fzc41M7PZDrHFgYu3atfaEj3YQp06dsifhShcQUYN+/fr16dPHntOmuFNUVLR79277t8zMQkQNAABvvfXWw4cPy8vLYRvyGJFIZE9GzK4h4npgQqFQjB8//tKlS7ANAdXV1QwGIyAgwEHtE3QcmJyCV69evW7dOrhm3Lp1a/Xq1Y4TgNAaAACSkpI8PDx++eUXiDY0Nzf/+OOPDu2C0BoAAN5+++26urqO+/4TJkxwaI8d/aiNRmNqaqpDu3MBDQAAK1asWLlypVwuBwAMGTKkra3t1q1bDupr4cKFra2tJhk2bNhw5MgRB3XUERfQAACwbdu2lJSU+Ph4FEVbWloc5J2n1WqbmpoAAK2trcOHD09OTp42bZojOnoK19Bg/vz5RqPR9G8EQRzkklRWVtZ+kkCj0SxYsMARvTyLC2gwduzYjofuMAyrqqpyREdlZWUtLU9CNmEYNmjQIGo9AAAAgUDg7v4ktAiCIDqdzhGeYffu3TNl9WmHzWY74SC7C2iwdevWRYsWRUREcDgc09mp1tbWBw/wTxjYriuGYUKhcOjQoRs3bnTCS1wXiG8qEAhmzZo1a9asv/766/Dhw9XV1S0tLXl5efYfxutIQ0ODVCpFEMTX13fQoEEZGRmRkZE4tt8FBHpX0fBI01CpbW3UKWUGBpMmk+jNFlOr1VKplEaj+fv742tAVWUlj8fjC/h0upmfJpNFQ2iAI6BzhQyfQLeQKA93D3xmEfgaiGu0uVdkD+8r3ThMrhcHoSEMNzrLjUGUn8Z/QBDEaDDqNUaD1oChaEuNQujL6pfA7zeMb2/LEDWQtxiu/CYR1+lEQQKejwed6QKLU0fUUq1GppFUSodN9I5OsF0JaBr8fU52/7rMs4dQ4M+BYgBeGHRoU3mzOxtMWuhvW4REOBqc3dPUIgF+vW3Pb0g0tCp92bWaVz7q4RVodZIuCBpcONgsk9FEwfZOowTk4Y2a6UuDOHzr7jadrcHJHQ0anZuoBwkFMFF+rXr6B8E8TytkcOoy+PfZVpWKTmIBAADhQ4P2fFNpVRXnaVBfoX1UovMOd+0cnN1CY9BC4vzP7LYi37nzNLj6h5jtScQs4LjjIXJvrNLVPbA0QpaTNHhUqNbpEI6IDFF9LcErzPPybxILCztJg7yrcq+eRJyFJM3VSz8bevfeGXyb9RC6IQxmTalFZxScoYFGhTZUqNl8Z4cChwuT7VaWZ1HQU2doUJGv5PngdmLCVeD5elTkW6SBM95di2t0PB9HrcbXbx25fG2vTN7kKQocEDt21LDZTKZbbV3J5q3z33j1uxNn/lXXUCoSBrw09p1+USNNVZSq1j9PfFdQfIXJcOsVNshBhrHYDL63e0uDztO/mwnAGeOgvlLjoPdxZy78/NfpzXExKdMmrYiNHnPp6u7Df35t+kiv1+4+8I+RSTMWvf6jSOi/99BnKpUUAKA36LbsWFxQdHlk0syXxr3T0urAYG5atdGSvBHOGAdquUHUE+c0IgAAmVx8/sqOWVO/iO032nRFwPM+cmzNxLQPTP876aUP42JSAABpKW9t/HHOg0d3Y6OTr2Ufqm8oWzDnh94RQwAAPXvErN00HXfbTNBZdJXM2G0xx2uAARqDxnTHv6OyB7eMRsOewyv3HF75pDMAZIrHz0cs5uOTtiJhAABArhADAPKLLgf4RZgEAADQaPj/ONphurM0aiJogACN0oAaUNynI7lCAgB4Y/YGocC343Uvz+CGxv/abWbQmQAAFDUCAKSyhqCAPvha0hl6rQGhdX836Iy5iM2lG7RG3DVgsx+/d/L16Wl5LS5HpFQ56QQuajBa8g7VGWuyB4+h1+Kc0ggAEBkejyBI1s2D7Ve0uu5fDwQF9KmuLWwSW/dazTYMOos0cMY4CAhzb2rEXwNvrx7DE6ZfvbH/l90fRkc9r1BIrt08/MarG4IDn+uiVvKIjL9zT/zrl4UjE2fwed45907jblg7qMHoGdD9lo4zNAiN8qgobBEF256loTMmpC4RCnyzsg+VlGfzed79+o4S8H27ruLtFTw/4/vjpzedvvCzUOAXEzWqtNwhkXtUrRoPHt2N3X1sFyft4fz40YM+z4fS6K4dxdEqmspbIvsxBiR3n2fDST5efRMEUrG6i+37oye/v5VjJmBucMBzNfXFZqssnr/VzzcMLwtPnP3X9VtmPN2ZDDe9QWu2ymcfHXdjdRpoBtXpIwdY9JrSSeOgTWnc9VVl75GdxoVWqqQ6nZm3jAjSqYUCvq9ZZyzbUKllWq2Z1zsGg57BMJ+EWSQM6CyMVEu13FNkGPX/LPJVdd5+8pXfJBIJzbOHwDndwaXw/KMFX4czmBbNvc7bRxs52VsrU5MgLXO3SGukwyZ6WyiAc/f0EZA21+/hTTLk/ugCab3S3c3Qf4QVw92pfhUCb+bo6T7VuQ3O7NSZSBtUBoXqxQzrIm0428UzLJozeprXo9sun/7jWVprFWqxbOp7ViezhuPr2FCpObKppufAAI4nKXb5MSCpkvE8DOMyunlCNAs0n1+jATv2c4NChnqHebr0VrPkkbShtPX5qb4xtjrBQz5/UF3advV3CUBo7gI2x8vDjWP+TpyAyJvUbVIVgqJBEW7DxtvlvAz/DAgAoL5C8/C+sjxPxWIztG1GOovB8mAZdN3vfjgTOo1mNBiMeqNBa2SyEKEvMyKW0yuWy2Lbu6YSQoN2VFKjSmFQyw3aNlSnJVbqNDodYbAQDp/BETAEXkwEv7sZYmnwv4mLHT8iJZQG8KE0gA+lAXwoDeBDaQCf/w+m2DnsA9UFyQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<IPython.core.display.Image object>"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "for s in app.stream({\"task\": task}):\n",
        "    print(s)\n",
        "    print(\"---\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "p9LGZ9K4uH63",
        "outputId": "7d22438d-1408-488d-c032-c37b5323ccb8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'plan': {'steps': [(\"Search for the winner of the 2024 Men's Australian Open.\", '#E1', 'websearch', '\"2024 Men\\'s Australian Open winner\"'), (\"Retrieve details about the winner's hometown using the search result from #E1.\", '#E2', 'LLM', '\"Find the hometown of #E1, the 2024 Men\\'s Australian Open winner\"')], 'plan_string': 'To solve this task, we need to find out the winner of the 2024 Men\\'s Australian Open and then determine their exact hometown. Here\\'s how we can approach it step-by-step:\\n\\nPlan: Search for the winner of the 2024 Men\\'s Australian Open.\\n#E1 = websearch[\"2024 Men\\'s Australian Open winner\"]\\n\\nPlan: Retrieve details about the winner\\'s hometown using the search result from #E1.\\n#E2 = LLM[\"Find the hometown of #E1, the 2024 Men\\'s Australian Open winner\"]\\n\\nNow, with these steps, we will be able to identify both the winner\\'s name and their exact hometown.'}}\n",
            "---\n",
            "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-19-23.pdf\\', \\'content\\': \"2024 MEN\\'S AUSTRALIAN OPEN WINNER. PLAYER VS FIELD ODDS. FIELD. PLAYER. EV 42202 -130 JANNIK SINNER. 42203 +365 42204 -470 DANIIL MEDVEDEV.\"}, {\\'url\\': \\'https://vsin.com/wp-content/uploads/2024/01/circa-sheets-1-21-24.pdf\\', \\'content\\': \"2024 MEN\\'S AUSTRALIAN OPEN WINNER. PLAYER. 42201. 42203. PLAYER VS FIELD ODDS. FIELD. EV 42202 -130 JANNIK SINNER. +365 42204 -470 DANIIL\"}, {\\'url\\': \\'https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/\\', \\'content\\': \\'[](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-1)Plan: Use Google to search for the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-4)Plan: Retrieve the name of the 2024 Australian Open winner from the search results. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-7)Plan: Use Google to search for the hometown of the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-1){\\\\\\'plan\\\\\\': {\\\\\\'plan_string\\\\\\': \"Plan: Use Google to search for the 2024 Men\\\\\\'s Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-3){\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"Qinwen Zheng, 6-3, 6-2\\\\\\\\\\\\\\\\nOur Latest Tennis Stories\\\\\\\\\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\\\\\\\\\n2024 Australian Open odds, Sinner vs. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-5){\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"Qinwen Zheng, 6-3, 6-2\\\\\\\\\\\\\\\\nOur Latest Tennis Stories\\\\\\\\\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\\\\\\\\\n2024 Australian Open odds, Sinner vs.\\'}, {\\'url\\': \\'https://blog.csdn.net/qq_41472205/article/details/144353195\\', \\'content\\': \\'\\\\\\', \\\\\\'#E3\\\\\\', \\\\\\'Google\\\\\\', \\\\\\'\"[Winner\\\\\\\\\\\\\\'s Name\\\\\\')], \\\\\\'plan_string\\\\\\': \\\\\\'Plan: Search for the winner of the 2024 Men\\\\\\\\\\\\\\'s Australian Open tennis tournament. {\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\\\\\\\\'s Australian Open winner]\\\\\\\\\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown\"}, {\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\\\\\\\\'s Australian Open winner]\\\\\\\\\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown\"}, {\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\\\\\\\\'s Australian Open winner]\\\\\\\\\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown\"}, {\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER.\\'}, {\\'url\\': \\'https://www.bets.com.au/tennis/australian-open/2024-mens-australian-open-winner-betting-odds-20230206-0009/\\', \\'content\\': \"Australian Open 2025 Tips, Predictions & Live Streaming Australian Open 2025 Betting Tips, Predictions & Live Streaming ### Aus Open Jannik Sinner vs Alexander Zverev Tips, Prediction & Live Stream - Sinner to defend Australian Open title ### Aus Open Aryna Sabalenka vs Madison Keys Tips, Prediction & Live Stream - Keys to defy the odds and win first Grand Slam title Our team of tennis experts at Bets.com.au provides betting tips, match previews and live streaming guides for the 2025 Australian Open. 2025 Women\\'s Australian Open Betting Odds Betting On The Australian Open Online bookmaker\\\\xa0bet365\\\\xa0offer their customers a comprehensive live streaming service, which broadcasts Grand Slam tennis all year - including the Australian Open. ATP Tour Australian Open\"}]'}}}\n",
            "---\n",
            "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-19-23.pdf\\', \\'content\\': \"2024 MEN\\'S AUSTRALIAN OPEN WINNER. PLAYER VS FIELD ODDS. FIELD. PLAYER. EV 42202 -130 JANNIK SINNER. 42203 +365 42204 -470 DANIIL MEDVEDEV.\"}, {\\'url\\': \\'https://vsin.com/wp-content/uploads/2024/01/circa-sheets-1-21-24.pdf\\', \\'content\\': \"2024 MEN\\'S AUSTRALIAN OPEN WINNER. PLAYER. 42201. 42203. PLAYER VS FIELD ODDS. FIELD. EV 42202 -130 JANNIK SINNER. +365 42204 -470 DANIIL\"}, {\\'url\\': \\'https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/\\', \\'content\\': \\'[](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-1)Plan: Use Google to search for the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-4)Plan: Retrieve the name of the 2024 Australian Open winner from the search results. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-9-7)Plan: Use Google to search for the hometown of the 2024 Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-1){\\\\\\'plan\\\\\\': {\\\\\\'plan_string\\\\\\': \"Plan: Use Google to search for the 2024 Men\\\\\\'s Australian Open winner. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-3){\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"Qinwen Zheng, 6-3, 6-2\\\\\\\\\\\\\\\\nOur Latest Tennis Stories\\\\\\\\\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\\\\\\\\\n2024 Australian Open odds, Sinner vs. [](https://langchain-ai.github.io/langgraph/tutorials/rewoo/rewoo/#__codelineno-17-5){\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"Qinwen Zheng, 6-3, 6-2\\\\\\\\\\\\\\\\nOur Latest Tennis Stories\\\\\\\\\\\\\\\\nSinner, Sabalenka win Australian Open singles titles\\\\\\\\\\\\\\\\nSinner makes epic comeback to win Australian Open\\\\\\\\\\\\\\\\n2024 Australian Open odds, Sinner vs.\\'}, {\\'url\\': \\'https://blog.csdn.net/qq_41472205/article/details/144353195\\', \\'content\\': \\'\\\\\\', \\\\\\'#E3\\\\\\', \\\\\\'Google\\\\\\', \\\\\\'\"[Winner\\\\\\\\\\\\\\'s Name\\\\\\')], \\\\\\'plan_string\\\\\\': \\\\\\'Plan: Search for the winner of the 2024 Men\\\\\\\\\\\\\\'s Australian Open tennis tournament. {\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\\\\\\\\'s Australian Open winner]\\\\\\\\\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown\"}, {\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\\\\\\\\'s Australian Open winner]\\\\\\\\\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown\"}, {\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. {\\\\\\'tool\\\\\\': {\\\\\\'results\\\\\\': {\\\\\\'#E1\\\\\\': \\\\\\'[{\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/Circa-Betting-Sheets-1-18-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER. #E1 = Google[2024 Men\\\\\\\\\\\\\\'s Australian Open winner]\\\\\\\\\\\\\\\\nPlan: Once the winner is identified, search for their exact hometown\"}, {\\\\\\\\\\\\\\'url\\\\\\\\\\\\\\': \\\\\\\\\\\\\\'https://vsin.com/wp-content/uploads/2024/01/circa-betting-sheets-1-22-24.pdf\\\\\\\\\\\\\\', \\\\\\\\\\\\\\'content\\\\\\\\\\\\\\': \"2024 MEN\\\\\\\\\\\\\\'S AUSTRALIAN OPEN WINNER.\\'}, {\\'url\\': \\'https://www.bets.com.au/tennis/australian-open/2024-mens-australian-open-winner-betting-odds-20230206-0009/\\', \\'content\\': \"Australian Open 2025 Tips, Predictions & Live Streaming Australian Open 2025 Betting Tips, Predictions & Live Streaming ### Aus Open Jannik Sinner vs Alexander Zverev Tips, Prediction & Live Stream - Sinner to defend Australian Open title ### Aus Open Aryna Sabalenka vs Madison Keys Tips, Prediction & Live Stream - Keys to defy the odds and win first Grand Slam title Our team of tennis experts at Bets.com.au provides betting tips, match previews and live streaming guides for the 2025 Australian Open. 2025 Women\\'s Australian Open Betting Odds Betting On The Australian Open Online bookmaker\\\\xa0bet365\\\\xa0offer their customers a comprehensive live streaming service, which broadcasts Grand Slam tennis all year - including the Australian Open. ATP Tour Australian Open\"}]', '#E2': 'content=\"Jannik Sinner won the 2024 Men\\'s Australian Open. He is from San Candido, Italy.\" additional_kwargs={\\'refusal\\': None} response_metadata={\\'token_usage\\': {\\'completion_tokens\\': 23, \\'prompt_tokens\\': 1527, \\'total_tokens\\': 1550, \\'completion_tokens_details\\': {\\'accepted_prediction_tokens\\': 0, \\'audio_tokens\\': 0, \\'reasoning_tokens\\': 0, \\'rejected_prediction_tokens\\': 0}, \\'prompt_tokens_details\\': {\\'audio_tokens\\': 0, \\'cached_tokens\\': 0}}, \\'model_name\\': \\'gpt-4o-2024-08-06\\', \\'system_fingerprint\\': \\'fp_523b9b6e5f\\', \\'finish_reason\\': \\'stop\\', \\'logprobs\\': None} id=\\'run-90236759-f030-4f91-a108-6d89e34bf0c6-0\\' usage_metadata={\\'input_tokens\\': 1527, \\'output_tokens\\': 23, \\'total_tokens\\': 1550, \\'input_token_details\\': {\\'audio\\': 0, \\'cache_read\\': 0}, \\'output_token_details\\': {\\'audio\\': 0, \\'reasoning\\': 0}}'}}}\n",
            "---\n",
            "{'solve': {'result': 'San Candido, Italy.'}}\n",
            "---\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "task1 = 'What is the name of the cognac house that makes the main ingredient in The Hennchata?'"
      ],
      "metadata": {
        "id": "0KbQ9BUQuKPH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for s in app.stream({\"task\": task1}):\n",
        "    print(s)\n",
        "    print(\"---\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TjzRX-NwuMbG",
        "outputId": "29437701-28d4-4adf-a33e-6b5e88a24c9a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'plan': {'steps': [('Identify the main ingredient in The Hennchata utilizing general awareness about the drink.', '#E1', 'LLM', 'What is the main ingredient in The Hennchata?'), ('Once the main ingredient is identified, find out which cognac house produces this ingredient by using a web search.', '#E2', 'websearch', 'Which cognac house makes #E1?')], 'plan_string': 'To determine the cognac house responsible for making the main ingredient in The Hennchata, I will begin by understanding what the main ingredient in The Hennchata is. From this, I can identify the cognac house.\\n\\nPlan: Identify the main ingredient in The Hennchata utilizing general awareness about the drink.\\n#E1 = LLM[What is the main ingredient in The Hennchata?]\\n\\nPlan: Once the main ingredient is identified, find out which cognac house produces this ingredient by using a web search.\\n#E2 = websearch[Which cognac house makes #E1?]'}}\n",
            "---\n",
            "{'tool': {'results': {'#E1': 'content=\\'The main ingredients in \"The Hennchata\" are typically Hennessy cognac and horchata. This cocktail combines the flavors of the smooth, bold cognac with the sweet, creamy, and spiced rice drink, horchata. Some variations of the drink might include additional spices or sweeteners, but Hennessy and horchata are the key components.\\' additional_kwargs={\\'refusal\\': None} response_metadata={\\'token_usage\\': {\\'completion_tokens\\': 78, \\'prompt_tokens\\': 19, \\'total_tokens\\': 97, \\'completion_tokens_details\\': {\\'accepted_prediction_tokens\\': 0, \\'audio_tokens\\': 0, \\'reasoning_tokens\\': 0, \\'rejected_prediction_tokens\\': 0}, \\'prompt_tokens_details\\': {\\'audio_tokens\\': 0, \\'cached_tokens\\': 0}}, \\'model_name\\': \\'gpt-4o-2024-08-06\\', \\'system_fingerprint\\': \\'fp_50cad350e4\\', \\'finish_reason\\': \\'stop\\', \\'logprobs\\': None} id=\\'run-29d0f354-7f44-4573-8a89-3cc6ee70aed5-0\\' usage_metadata={\\'input_tokens\\': 19, \\'output_tokens\\': 78, \\'total_tokens\\': 97, \\'input_token_details\\': {\\'audio\\': 0, \\'cache_read\\': 0}, \\'output_token_details\\': {\\'audio\\': 0, \\'reasoning\\': 0}}'}}}\n",
            "---\n",
            "{'tool': {'results': {'#E1': 'content=\\'The main ingredients in \"The Hennchata\" are typically Hennessy cognac and horchata. This cocktail combines the flavors of the smooth, bold cognac with the sweet, creamy, and spiced rice drink, horchata. Some variations of the drink might include additional spices or sweeteners, but Hennessy and horchata are the key components.\\' additional_kwargs={\\'refusal\\': None} response_metadata={\\'token_usage\\': {\\'completion_tokens\\': 78, \\'prompt_tokens\\': 19, \\'total_tokens\\': 97, \\'completion_tokens_details\\': {\\'accepted_prediction_tokens\\': 0, \\'audio_tokens\\': 0, \\'reasoning_tokens\\': 0, \\'rejected_prediction_tokens\\': 0}, \\'prompt_tokens_details\\': {\\'audio_tokens\\': 0, \\'cached_tokens\\': 0}}, \\'model_name\\': \\'gpt-4o-2024-08-06\\', \\'system_fingerprint\\': \\'fp_50cad350e4\\', \\'finish_reason\\': \\'stop\\', \\'logprobs\\': None} id=\\'run-29d0f354-7f44-4573-8a89-3cc6ee70aed5-0\\' usage_metadata={\\'input_tokens\\': 19, \\'output_tokens\\': 78, \\'total_tokens\\': 97, \\'input_token_details\\': {\\'audio\\': 0, \\'cache_read\\': 0}, \\'output_token_details\\': {\\'audio\\': 0, \\'reasoning\\': 0}}', '#E2': \"HTTPError('400 Client Error: Bad Request for url: https://api.tavily.com/search')\"}}}\n",
            "---\n",
            "{'solve': {'result': 'Hennessy'}}\n",
            "---\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "task2 = 'What is captial of UK'"
      ],
      "metadata": {
        "id": "GL_na_eJuPaC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for s in app.stream({\"task\": task2}):\n",
        "    print(s)\n",
        "    print(\"---\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1wKPz9jwuRZ2",
        "outputId": "5790ebc9-5dc1-41b2-ea70-fa126adfb62d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'plan': {'steps': [('To find the capital of the United Kingdom, use general world knowledge to provide a confident answer.', '#E1', 'LLM', 'What is the capital of the United Kingdom?')], 'plan_string': 'Plan: To find the capital of the United Kingdom, use general world knowledge to provide a confident answer.\\n#E1 = LLM[What is the capital of the United Kingdom?]'}}\n",
            "---\n",
            "{'tool': {'results': {'#E1': \"content='The capital of the United Kingdom is London.' additional_kwargs={'refusal': None} response_metadata={'token_usage': {'completion_tokens': 10, 'prompt_tokens': 16, 'total_tokens': 26, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-2024-08-06', 'system_fingerprint': 'fp_076d6600c1', 'finish_reason': 'stop', 'logprobs': None} id='run-6cfca7e5-eb02-4478-b251-765dbf7ed925-0' usage_metadata={'input_tokens': 16, 'output_tokens': 10, 'total_tokens': 26, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}\"}}}\n",
            "---\n",
            "{'solve': {'result': 'London'}}\n",
            "---\n"
          ]
        }
      ]
    }
  ]
}